### Bellman equation

Go back to the [[AI Glossary]]

#rl

In reinforcement learning, the following identity satisfied by the optimal Q-function:

![The Q-function in reinforcement learning](https://i.imgur.com/NgAUXxP.png)

Reinforcement learning algorithms apply this identity to create Q-learning via the following update rule:

![The Bellman equation](https://i.imgur.com/rAaoDsK.png)

Beyond reinforcement learning, the Bellman equation has applications to dynamic programming. See the Wikipedia entry for Bellman Equation.